Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Tobi Dance #22

Draft
wants to merge 23 commits into
base: master
Choose a base branch
from
Draft

Draft: Tobi Dance #22

wants to merge 23 commits into from

Conversation

frehe
Copy link
Member

@frehe frehe commented Dec 14, 2022

No description provided.

…mass M to m_cart and cart pole mass m to m_pole

added comments about meaning of "stage" in MPPI
renamed TargetPositionGenerator to TargetTrajectoryGenerator, since we target a complete trajectory, not just cart position.
renamed my_logger() to get_logger()
say which compiler we are using
…istency with other config files.

added more docstrings for undocumented methods and constructors.
added updated_attributes to barebones controller.
improved formatting of logging output
added dictdiffer to requirements.txt
renamed target_position cost function to cartpole_trajectory_cost.py.
replaced some load_config with the load_or_reload_config_if_modified, so that the configs are cached for change checking.
added logger to Compile.py
…g and the cartpole is at least balancing itself and swinging up from down position. Big changes all over!

Now anytime any of the config files is modified during runtime, the using class has its tf.variable assigned the new value.
To get this work, the variable MUST NOT EXIST before it is set this way the first time. Otherwise the compiler just uses the field at compile time and the changes are never seen inside.
The cartpole window now does not reset the initial position and angle sliders on each start.
…hanging the cost weights during runtime! Yay, finally!

Trick was to make sure that all values are actually propagated to tf variables, e.g. numpy arrays, ints, string, not just float

Changed logging format to start with level for better readability.
added string type to computation_library.py
Comment on lines +61 to +63
if not computation_library is None and computation_library not in self.predictor.supported_computation_libraries:
raise ValueError(
f"Predictor {self.predictor.__class__.__name__} does not support {computation_library.__name__}")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if not computation_library is None and computation_library not in self.predictor.supported_computation_libraries:
raise ValueError(
f"Predictor {self.predictor.__class__.__name__} does not support {computation_library.__name__}")
if computation_library is not None and computation_library not in self.predictor.supported_computation_libraries:
raise ValueError(
f"Predictor {self.predictor.__class__.__name__} does not support {computation_library.__name__}"
)

@@ -54,7 +54,7 @@ class ComputationLibrary:
gather: Callable[[TensorType, TensorType, int], TensorType] = None
gather_last: Callable[[TensorType, TensorType], TensorType] = None
arange: Callable[[Optional[NumericType], NumericType, Optional[NumericType]], TensorType] = None
zeros: Callable[["tuple[int]"], TensorType] = None
zeros: Callable[["tuple[int,...]"], TensorType] = None
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! I had wrong assumptions about how to type subscript tuples. Learned something again.


class TensorFlowLibrary(ComputationLibrary):
lib = 'TF'
reshape = tf.reshape
permute = tf.transpose
newaxis = tf.newaxis
shape = lambda x: x.get_shape() # .as_list()
shape = tf.shape # tobi does not understand reason for this previous definition: # lambda x: x.get_shape() # .as_list()
Copy link
Member Author

@frehe frehe Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No difference: If x is a tf.Tensor, then x.shape and x.get_shape() are identical https://www.tensorflow.org/api_docs/python/tf/Tensor#get_shape

Using tf.shape(x) seems to be the best choice here, which is a third option.
From https://www.tensorflow.org/api_docs/python/tf/shape:
tf.shape and Tensor.shape should be identical in eager mode. Within tf.function or within a compat.v1 context, not all dimensions may be known until execution time. Hence, when defining custom layers and models for graph mode, prefer the dynamic tf.shape(x) over the static x.shape.

@frehe
Copy link
Member Author

frehe commented Dec 14, 2022

I updated this branch with all recent changes of master and resolved the merge conflicts.

frehe and others added 15 commits December 15, 2022 10:27
# Conflicts:
#	src/SI_Toolkit/computation_library.py
added prefs to cartpole gui for initial position and angle
increased num_rollouts to 1000, decreased horizon to 25 time steps, changed cost function for spin to pure cart angle speed. Now spin, balance, and shimmy all work quite well.
moved cartpole_trajectory_generator back out outside compiled TF code for ease of development and debugging, it is cheap so this is OK.
added more logging of what gets compiled.
added time to arguments for the base controller so that contained methods can access current time.
added ruamel.yaml to requirements (this is the yaml parser that can handle scientific notation numbers)
# Conflicts:
#	Control_Toolkit_ASF_Template/Cost_Functions/EnvironmentName/cost_function_barebone.py
#	Control_Toolkit_ASF_Template/config_optimizers.yml
#	Controllers/__init__.py
#	Controllers/controller_neural_imitator_tf.py
#	Cost_Functions/__init__.py
#	Cost_Functions/cost_function_wrapper.py
#	Optimizers/optimizer_mppi.py
#	Optimizers/optimizer_rpgd_me_tf.py
#	Optimizers/optimizer_rpgd_ml_tf.py
#	Optimizers/optimizer_rpgd_particle_tf.py
#	Optimizers/optimizer_rpgd_tf.py
#	requirements.txt
# Conflicts:
#	src/SI_Toolkit/Predictors/predictor_wrapper.py
Started merging from remote/origin/Tobi_Dance
…cal-cartpole repo. Fix some cartpole parameter import. Fix rename of config_cost_functions.yml. Add timer.py and yes_or_no.py utility classes.
… seem to work. mppi parameters copied from development. cost weights adjusted.

controller update interval inccreased to 25ms from 20ms to match simulator and actual achieved rate with dancer mppi.
mppi rollouts reduced to 700 to speed up control.
PhysicalCartPoleDriver.py has class self variable for reference from other classes to the instance.
Compile.py raises exception when either option is undefined.
… for next timestep and recording this prediction along with the measurement in the PhysicalCartPoleDriver.py CSV output file. In this commit, control is enabled by default because the pycharm debugger hangs on interactive terminal keyboard intput, and the motor output is disabled for remote debugging.
commented out some noisy log.debug.
made horizon same as physical cartpole
updated satisfaction CSV to use markers from audition and added more cartsheels
…or for compatibilty with RPGD gradient computation.

update_attributes in globals_and_utils.py supports assignment of tf.Variable and now raises exception if we try to .assign an immutable Tensor.
Balance step now uses int +1 or -1 for desired pole up or down, not 'up' or 'down'.
computation_library.py now casts variale to numpy() from tensorflow only if it is a Tensor type.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants